Papers with true ranking
Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models exhibit positional bias in how they use context, which affects listwise ranking. |
| Approach: | They propose a method to marginalize out different list orders in the prompt to produce an order-independent ranking with less positional bias. |
| Outcome: | The proposed method improves on five datasets in sorting and passage reranking by 34-52% . it marginalizes out different list orders in the prompt to produce an order-independent ranking . |